\chapter{Functional testing}

\section{Introduction}

Our first task is to check the functional correctness of the application. The functional tests to be carried out are:
\begin{enumerate}
 \item Check each application configuration for the occurence of Java exceptions;
 \item Check each application configuration for any aggressive memory leaks i.e. any leaks that would cause the JVM to run out of memory if the application was left running.
\end{enumerate}

Any configurations that exhibit any of the above behaviour \footnote{an application 'fault'} will be discounted from any further performance testing. As far as the black-box methods that are available to us allow, we will investigate the extent and location of the fault 

\section{Exceptions}

\subsection*{Method}

To check for occurence of Java exceptions in our application we set up our application with some added monitoring. We used a profiler tool\footnote{provided by John Bergin - see apendices} that carried out instrumentation of specified classes (in our case those in package org.adaptivecellsj). The trace files produced by this tool gave us the calling sequence for these instrumented classes, enabling us to indentify the bean that raised the exception (in conjunction with the diagram in figure \ref{fig_bean_calling}. From the AdaptiveCells/J documentation we know that each bean calls a method, 'simulateBusinessLogic', we can see during which particular call to this method the exception is raised, thus identifying the bean e.g. if for config1 the exception is raised in the second call to simulateBusinessLogic then the bean that raised the exception is TB2.

We invoked each configuration, in sequence, through the web page front end, accessible at url http://localhost:8080/adaptivecellsj/start.html. If an exception was raised we examined the JBoss log, and the trace file produced by the profiling tool.

\subsection*{Results}

We established that three configurations raised exceptions\footnote{all of type java.lang.RuntimeException} config3, config5, and config9. Nothing of note was observed in the JBoss logs. Examination of the trace files showed that the following:

\begin{center}
\begin{tabular}{| c | c |}
 \hline
 Config & Bean raising exception \\
 \hline
 config3 & TB5 \\
 config5 & TB7 \\
 config9 & TB1 \\
 \hline
\end{tabular}
\end{center}

We repeated the test to check our results, which confirmed our initial findings. We can now discount config3, config5, and config9 from further testing.

\section{Memory Leaks}

\subsection*{Method}

Despite Java having garbage collection memory leaks can still occur, primarily where an object is put into a long lived collection and not taken out once the object is no longer required.

The instructions for the assignment state that we identify agressive memory leaks and remove them from further investigation. For our purposes we define an agressive memory leak as one that would result in heap exhaustion within standard use of the application i.e. the memory is not freed by garbage collection.

We know from the AdaptiveCells/J documentation that memory leaks are implemented by allocating an array of bytes. Any agressive memory leak should be visible as a large block of memory that isn't freed after a garbage collection. To identify configurations with agressive memory leaks we used the following method:

\begin{enumerate}
 \item Monitor the application with the Netbeans Profiler;
 \item Force a garbage collection, and take a snapshot of memory usage;
 \item Invoke the AdaptiveCells/J configuration that we are currently investigating, and then take a snapshot of memory usage;
 \item Repeat the last two steps until you've invoked the configuration five times;
 \item Force a garbage collection and take a final snapshot of memory usage;
 \item Examine the increase in memory taken by array of bytes (byte[]) from the snapshot taken before invoking the configuration and each snapshot taken after invoking the configuration i.e. establish a picture of how the memory leak was growing.
\end{enumerate}

The above method was repeated for each of the configurations that did not cause Java exceptions to be raised. An agressive memory leak should be evident from observing a steady increase in the memory allocated to byte arrays and that this allocation should not significantly decrease following the forced garbage collection.

\subsection*{Results}

We established that two configurations had agressive memory leaks, config6 and config8. Table \ref{func_mem_leak_table} shows the data obtained.

\begin{figure}[h]
 \centering
\begin{tabular}{| c | c | c |}
 \hline
  & \multicolumn{2}{|c|}{Config} \\ \cline{2-3}
 Stage & config6 & config8 \\
 \hline
 post invocation 1 & 1,703,904 & 1,034,504 \\
 post invoication 2 & 3,768,192 & 2,331,496 \\
 post invoication 3 & 5,704,304 & 3,250,352 \\
 post invoication 4 & 7,768,440 & 4,331,024 \\
 post invoication 5 & 10,029,840 & 5,250,432 \\
 post garbage collection & 9,768,488 & 5,140,432 \\
 \hline
\end{tabular}
 \caption{Change in memory (from initial reading) allocated to byte[]}
 \label{func_mem_leak_table}
\end{figure}

If we graph this data we can that following a forced garbage collection (at Invocation 6) there is minimal freeing of memory allocated to byte[]s, indicating that we have a memory leak. The amount of memory leaked would apprear to of the order of 2,000,000 bytes per invocation for config6, and 1,000,000 bytes for config8. This is backed up by looking at the position before any invocations and the position after a forced garbage collection with a leak of 9,768,488 bytes and 5,140,432 bytes respectively.

\begin{figure}[ht]
 \centering
 \scalebox{0.5}{\includegraphics{Graphics/func_memory_leaks.png}}
 % func_memory_leaks.png: 638x448 pixel, 72dpi, 22.51x15.80 cm, bb=0 0 638 448
 \caption{Graph showing memory leakage for config6 and config8}
 \label{func_mem_leak_graph}
\end{figure}

\subsection{Summary}

Following our functional testing we can discount config6, config8 because of agressive memory leaks and config3, config5, and config9 because of raising Java exceptions. This leaves config1, config2, config4, config7, and config10 remaining to carry out performance analysis on.